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ABSTRACT 

Advances  in  computer  graphics  hardware  have  led  to  astounding 
increases  in  visual  realism  for  3D  environments.  Although  this 
improved  visual  realism  can  lead  to  a  better  sense  of  immersion 
and  a  more  faithful  reproduction  of  the  natural  world,  it  does  not 
necessarily  promote  increased  awareness  of  an  evolving  situation. 
Although  some  requirements  overlap  with  other  applications  such 
as  off-the-shelf  games,  military  Command  and  Control  systems 
(including  stability  operations  and  homeland  defense  missions) 
have  unique  requirements.  In  particular,  the  overriding  goal  is  to 
provide  timely  data  effectively,  independent  of  how  visually 
pleasing  it  may  be.  The  primary  objective  is  to  convert  the 
abundance  of  information  into  concise  knowledge.  This  paper 
proposes  a  method  of  combining  realistic  and  non-realistic 
displays  to  increase  display  effectiveness,  and  hence  improve 
situational  awareness. 

CG  Keywords:  1.3.6  [Computer  Graphics]:  Methodology  and 
Techniques  -  Interaction  Techniques;  J.7.0  [Computer 
Applications]:  Computers  in  Other  Systems  -  Command  and 
Control;  1.3.7  [Computer  Graphics]:  Three-Dimensional  Graphics 
and  Realism  -  Virtual  Reality; 

Additional  Keywords:  information  visualization,  HCI  (Human- 
Computer  Interface),  Military  training  systems,  situational 
awareness. 


1.  INTRODUCTION 

There  are  many  visualization  techniques,  but  in  order  to  be  useful 
they  must  work  together  in  a  consistent,  unified  fashion.  A 
realistic  view  provides  a  more  natural  interface  for  viewing 
geographical  data  and  provides  the  optimal  domain  for  spatial 
reasoning.  However,  non-realistic  display  techniques  may  enable 
a  viewer  to  more  quickly  assess  an  overall  situation.  Our  goal  is 
to  combine  the  best  of  both  visualization  techniques,  along  with 
other  types  of  data  displays,  to  enhance  understanding  of  the  data. 
This  paper  discusses  military  display  systems  with  respect  to  the 
requirements,  visualization  techniques,  and  most  importantly, 
their  interaction  in  a  commercially  successful  product. 

An  effective  information  station  helps  the  user  to  make  better 
decisions,  in  a  timely  fashion.  Much  of  Command  &  Control, 
Communications,  Computers  and  Intelligence  (C4I)  deals  with 
making  decisions  in  the  face  of  uncertain  or  inadequate  data.  One 
of  the  more  important  aspects  of  making  those  decisions  is 
situational  awareness  -  the  ability  to  comprehend  the  people, 
places,  history  and  current  events  that  will  affect  the  battle  space. 
Reducing  uncertainty  (also  referred  to  as  the  'Tog  of  war"  or 
“friction”),  so  that  better  decisions  can  be  made  is  the  goal  of 
situational  awareness. 

“There  are  four  sources  of  ‘fog’  commanders  and  staffs  must 
overcome  to  achieve  accurate  situational  understanding: 

•  Inadequate  or  poor-quality  information. 

•  Misinterpretation  of  information. 


•  Conflicting  information  or  choices. 

•  Too  much  information.”  [1] 


"When  commanders’  situational  understanding  is  better  than  their 
enemy’s,  they  have  a  significant  but  temporary  advantage."  [2] 

An  information  station  that  can  be  used  to  simulate  and  train 
military  operations  has  several  components.  These  may  include  a 
2D  display,  a  3D  display,  text  and  hierarchical  lists,  etc.  This 
paper  focuses  on  the  3D  display  and  to  a  minor  extent,  its 
interaction  with  the  other  types  of  displays. 

In  the  following  sections  we  discuss  the  requirements  of  a 
military  training  system,  categorize  the  non-realistic  data  we 
visualize,  discuss  specific  types  of  data  where  a  non-realistic 
version  is  more  effective  than  a  realistic  version  and  how  the 
assorted  visualizations  interact  in  a  complex  system.  We 
conclude  with  a  discussion  of  related  products. 

2.  REQUIREMENTS 

Our  list  of  requirements  is  based  on  feedback  from  fielded 
systems  with  previous  versions  of  this  and  other  products, 
customers  (both  foreign  and  domestic)  who  purchase  the  base 
toolkits  in  order  to  build  their  own  systems,  and  employees  with 
military  experience.  Some  of  the  fielded  system  locations 
include:  Fort  Leavenworth  Command  and  General  Staff  College; 
Fort  Lee  (logistics  course);  USMA  (West  Point);  U.S.  National 
Guard  35  Field  Training  Group;  Illinois  National  Guard;  USME 
Expeditionary  Warfare  School  in  Quantico,  UA. 

Military  training  systems  have  many  similarities  to  commercial 
games,  in  particular  those  with  a  military  flare  like  America’s 
Army,  Full  Spectrum  Warrior  or  Close  Combat.  Dynamically 
changing  data  forces  the  user  (player)  to  make  time  critical 
decisions.  The  scenario  (game)  progresses,  even  without  active 
user  input,  so  that  lack  of  an  active  decision  is  still  an  action. 

However,  there  are  a  substantial  number  of  important  differences, 
the  most  important  of  which  is  the  primary  goal  —  not  to  entertain, 
but  to  train.  According  to  Alberts,  Garstka  and  Stein,  “Providing 
battlespace  awareness  ...  with  requisite  accuracy  and  timeliness 
requires  that  data  and  information  from  multiple  sources  be  ... 
presented  in  ways  that  facilitate  rapid  and  accurate  inferences.”[3] 
Realistic  displays  are  not  bad,  but  do  not  necessarily  lead  to 
improved  situational  awareness.  This  is  counter-intuitive,  as  most 
people  prefer  the  most  realistic  displays,  even  though  their 
performance  might  be  worse  [4]. 

A  series  of  research  experiments  by  people  like  Smallman,  St. 
John  and  Cowen  have  shown  2D  is  better  for  some  tasks,  while 
3D  is  better  for  others  [4].  Nagata  showed  that  increasing  the 
depth  cues  in  3D  displays  increases  localization  [5]. 

Similarly,  many  advocate  the  use  of  non-realistic  visualizations  to 
improve  various  aspects  performance  in  3D  environments.  By 
blending  the  most  useful  aspects  of  2D  and  3D,  realistic  and  non- 
realistic  displays,  we  can  create  a  product  that  takes  advantage  of 
the  best  of  each.  To  reiterate,  realism  is  not  more  important  than 
accurate  assessment  of  data. 

Some  of  the  more  substantial  differences  between  a  shrink- 
wrapped  game  and  a  tactical  training  system  are  the  range  and 
sophistication  of  users;  when  and  why  data  is  developed;  playing 
time  evaluation  metrics,  and  the  upgrade  path  of  the  application. 

Users  at  different  levels  of  command  and/or  in  different  roles 
must  work  together.  The  data  must  be  synchronized,  but  differ 
according  to  individual  needs.  The  training  system  must  be 
flexible  enough  to  support,  not  only  different  roles,  but  the  ability 


to  dynamically  switch  between  them  as  users  switch  roles  to 
cover  someone  who  is  no  longer  able  to  fulfill  that  role,  or  to 
allow  commanders  to  view  what  their  staff  officers  are  seeing. 

Unlike  games,  there  is  no  progression  through  levels  where  users 
learn  interaction  techniques;  all  techniques  could  potentially  be 
used  at  any  point  in  time.  (Not  a  different  technique  per  "level"). 
Nor  are  the  users  necessarily  familiar  with  similar  products.  This 
makes  the  training  system  interfaces  harder  to  learn.  As 
compensation,  the  user  base  accepts  that  some  user  training  is 
necessary,  unlike  games  installed  by  isolated  individuals  on  their 
home  machines. 

Generation  of  the  source  data  (scenarios)  is  substantially  different 
than  games.  Scenario  generators  (instructors)  generate  the  data  to 
meet  particular  training  objectives,  after  the  training  system  is 
developed.  Based  on  the  results  from  one  training  exercise,  a  new 
scenario  can  be  created  to  meet  the  next  training  objectives.  This 
is  in  contrast  to  games,  where  the  game  designers  have  control 
over  the  input  data,  and  the  data  is  created  before  the  game  is 
shipped. 

Since  the  data  is  based  on  a  realistic  scenario,  as  opposed  to  what 
would  make  the  game  most  interesting  to  play,  too  much  data  is 
more  common  than  too  little  data.  Consequently,  de-cluttering 
the  data  is  an  important  task  -  tying  directly  back  to  the  goal  of 
reducing  the  fog  of  war. 

Training  scenarios  are  typically  long  endeavors.  There  are  many 
phases  preceding  the  battle  such  as  planning  and  deployment. 
Various  brigade/battalion  training  scenarios  run  at  Fort 
Leavenworth  took  between  1  and  4  hours  from  start  to  finish  [6]. 
These  scenarios  were  predominantly  using  2D  displays,  but  we 
anticipate  similar  time  commitments  with  3D  displays 

In  order  to  provide  effective  feedback,  different  evaluation 
metrics  are  required  than  those  provided  by  first  person  shooting 
games.  Detailed  after  action  reviews  require  detailed  data 
histories  of  vehicle  movements,  interactions,  status  changes  etc., 
as  well  as  the  ability  to  display  and  discuss  these  concepts  within 
a  group.  We  address  this  issue  by  using  a  separate  product  which 
records  the  data  for  subsequent  analysis  and  replay,  and  providing 
"instructor  mode"  view  controls  so  that  one  machine  can  control 
the  views  of  others.  This  recorded  data  can  be  replayed  similarly 
to  a  VCR,  with  override  capabilities  for  point  of  view. 

3.  APPROACH 

In  order  to  aid  the  user  in  the  important  goal  of  understanding  the 
data,  we  implemented  a  series  of  non-realistic  techniques  that 
augment  the  realistic  virtual  display.  This  could  be  considered 
"augmented  virtuality"  or  "exaggerated  reality".  First  we  will 
discuss  the  non-realistic  techniques,  and  then  their  interaction 
with  the  realistic  techniques. 

The  non-realistic  techniques  can  be  categorized  as  follows: 

•  Visualization  of  non-visual  data.  Additional  information  is 
given  a  visual  representation. 

•  Alternative  visualization.  A  non-realistic  visualization 
technique  is  used  in  place  of  a  realistic  depiction. 

•  Data  reduction.  Data  that  would  normally  be  displayed  in  a 
realistic  view  is  removed  from  view  or  de-emphasized  so 
that  the  focus  is  on  the  relevant  information.  This  is  also 
referred  to  as  "de-cluttering". 


3.1  Visualization  of  Non-Visual  Data 

Some  types  of  data  are  invisible,  so  no  realistic  depiction  is  even 
possible,  such  as  the  volume  swept  out  by  a  radar  sensor.  Pictures 
show  the  relationship  between  objects  (“is  this  entity  within  the 
coverage  of  that  sensor?”  “What  areas  of  the  battlefield  have  no 
coverage?”  etc.)  Translucent  sensor  volumes  highlight  the  area 
being  sensed,  but  do  not  obscure  other  data.  User  controlled  color 
settings  can  be  used  to  distinguish  between  various  types  of 
sensor  coverage  if  desired.  This  is  important,  as  sensors  differ  in 
frequency  and  accuracy,  i.e.  the  types  of  things  they  can  “see”. 
Visualization  of  this  type  of  data  concisely  shows  how  the  data  is 
changing,  as  the  sensor  moves  relative  to  the  both  the  terrain  and 
other  entities. 

Several  criteria  are  used  to  select  which  non-visual  data  to  depict, 
and  how  to  depict  it.  The  data  must  be  relevant  to  the  user,  help 
the  user  with  his/her  task,  and  most  importantly,  have  a  spatial 
representation.  An  example  of  non-visual  data  without  a  spatial 
organization  is  the  force  hierarchy.  In  this  case,  a  list  box  is  a 
better  representation. 

Some  data  is  important  to  visualize  because  its  spatial  location 
changes  rapidly  over  time.  This  includes  threat  domes  (the  area 
surrounding  an  entity  for  which  its  weapons  are  effective, 
communication  lines  (which  can  rapidly  change  due  to  relative 
changes  in  location,  terrain  interference,  or  weather  interference; 
biological,  chemical,  and  nuclear  clouds  (which  can  be  color 
coded  to  indicate  type  of  hazard),  drop  down  lines  with  altitude 
labels  for  planes  which  indicate  height  above  terrain  and  the 
precise  XY  location  (which  may  not  be  obvious  in  a  perspective 
view),  movement  or  trajectory  history  for  vehicles  and  missiles 
(which  help  in  prediction  of  future  location  and  analysis  during 
after  action  reviews),  inter-visibility  lines  etc. 

Other  data  does  not  change  location,  but  is  important  for  planning 
and  decision-making.  Tactical  graphics  such  as  planned  routes, 
waypoints,  areas  of  impassible  terrain  and  minefields  are  all 
examples  of  non- visual  data  that  are  represented  visually. 

Once  we  determine  what  data  should  be  represented,  it  must  be 
decided  which  are  the  important  aspects  to  represent,  how  to 
represent  them  and  for  how  long. 

History  movement  is  conceptually  simple,  and  hence  a  good 
example.  The  “what”  is  easy  -  where  did  the  vehicles  come  from. 
How  to  represent  this  has  many  options.  A  series  of  ghost  vehicle 
images  shows  clearly  where  the  vehicle  used  to  be,  but  due  to 
their  size,  clutter  the  display.  A  single  line  drawn  at  the  center 
point  of  the  vehicle  will  show  the  past  location,  but  not  the 
vehicles  orientation  (more  important  for  submarines  and  planes 
than  tanks).  But  a  “ribbon”  will  show  both.  Since  there  are  many 
vehicles,  color-coding  the  history  trail  by  force  affiliation  reduces 
confusion. 

The  last  consideration  for  deciding  on  the  visual  representation  of 
non- visual  data  is  how  long  it  should  be  displayed.  Vehicle 
trajectory  histories  show  the  movement  of  any  vehicle,  and  can  be 
turned  on  or  off  though  a  dialog.  This  balances  information 
availability  and  screen  clutter.  There  are  two  options  for  the 
length  of  the  history  trail:  distance  or  time.  Each  has  its  own 
advantages.  With  time,  the  user  configures  the  relevant  time 
period  —  say  N  minutes.  The  history  trail  shows  where  the 
vehicle  was  during  the  last  N  minutes.  Since  different  types  of 
vehicles  (such  as  planes  and  tanks),  move  at  such  substantially 


different  speeds,  the  length  of  the  trail  is  an  indicator  of  velocity 
(faster  vehicles  have  longer  trails).  Using  distance,  uniformly 
spaced  locations  are  saved.  All  vehicle  trails  are  the  same  length, 
so  no  velocity  information  is  available.  There  is  potentially  less 
screen  clutter  with  distance  based  history  trails  than  time  based. 
Given  the  same  number  of  data  samples,  the  distance-based 
approach  will  give  finer  granularity  for  abrupt  changes  in 
orientation  and  direction,  such  as  loops. 


Figure  2  "Ribbons”  show  the  location  and  orientation  history. 

3.2  Alternative  Visualization 

According  to  Booch  et.  al,  “A  rendering  is  an  abstraction  that 
favors,  preserves,  or  even  emphasizes  some  qualities  while 
sacrificing,  suppressing,  or  omitting  other  characteristics  that  are 
not  the  focus  of  attention.”[7]  Raskar  and  Cohen  compare  photo¬ 
realistic  to  non-photo-realistic  rendering  and  state  that  “a 
simplified  diagram  is  often  preferred  when  an  image  is  required  to 
delineate  and  explain”  [8].  Alternative  visualizations  are  used 
when  a  realistic  depiction  is  possible  but  is  less  helpful  to  the  task 
at  hand  than  a  non-realistic  or  rendered  version. 

Of  the  graphical  models  used  in  a  military  training  system,  those 
used  to  represent  vehicles  are  the  most  pervasive,  so  we  spend  the 
most  time  talking  about  the  visualization  of  these,  followed 
briefly  by  an  overview  of  a  few  of  the  other  objects  for  which  we 
support  alternative  visualizations. 

3.2.1  Vehicles 

Dynamic  vehicle  model  swapping  supports  realistic,  non-realistic, 
and  symbolic  models;  shown  respectively  in  Figure  1  (a,  b,  c). 
Each  model  category  offers  distinct  tradeoffs  in  the  task  of 
vehicle  identification. 

For  all  vehicle  model  categories,  shape  identifies  whether  the 
entity  is  a  land,  air  or  sea  vehicle,  and  the  models  come  in  various 
levels  of  detail.  The  first  category,  realistic  models,  is  quite 
standard  in  3D  environments  and  need  not  be  discussed  in  detail 
here. 

The  second  category,  non-photo-realistic  vehicle  models,  is 
reminiscent  of  cartoon  images.  Color  is  used  to  emphasize  the 
force  affiliation  (friendly,  enemy,  neutral  or  unknown).  The 
shape  provides  a  coarse  categorization  of  vehicle  type  (land,  air  or 
sea).  These  models  lose  the  realistic  rendering  that  would,  for 
example,  allow  one  to  distinguish  the  exact  type  of  aircraft.  Due 
to  their  simplified  form,  the  non-realistic  models  have  fewer 
polygons,  and  are  faster  to  render,  which  means  that  scenarios 
with  larger  numbers  of  entities  can  be  supported  without  loss  of 
performance. 


The  third  vehicle  model  category,  symbolic  billboards,  is  the  least 
realistic,  but  provides  stylized  information.  A  configuration  file 
maps  the  vehicle  type  to  any  graphic  desired.  In  our  application, 
we  use  standard  military  symbology  2525A/B.  Like  the  non- 
realistic  models,  color  indicates  force  affiliation.  The  symbol 
markings  allow  the  knowledgeable  user  to  recognize  the  exact 
vehicle  type. 

Each  of  these  three  techniques  offers  different  trade-offs.  No  one 
approach  is  best  all  of  the  time.  By  allowing  the  user  to 
dynamically  switch  vehicle  models,  we  support  the  “most 
effective  view  at  the  right  time”. 

Realistic  models,  non-realistic  models,  and  symbolic  models 
show  progressively  less  realism  and  progressively  faster 
performance.  Compared  to  other  things,  such  as  terrain  models, 
the  cost  of  vehicle  models  is  small,  but  with  large  complex 
systems,  sometimes  every  little  bit  helps. 

As  with  realistic  models,  the  symbolic  billboards  show  the  exact 
vehicle  type,  but  need  a  smaller  amount  of  screen  real  estate  to  do 
so.  Realistic  models  in  the  distance  are  not  detailed  enough  to 
identify.  Although  the  user  can  increase  the  scale  at  which  the 
non-realistic  models  are  rendered  based  on  a  slider,  it  is  not 
always  feasible.  For  example,  when  the  view  is  further  away,  and 
a  large  number  of  vehicles  are  in  sight,  increasing  the  model  scale 
causes  the  vehicle  images  to  overlap.  In  this  situation,  symbolic 
billboards  are  superior  to  either  of  the  other  model  categories. 

This  is  consistent  with  experimental  results  by  Smallman  et  al. 
who  state,  “Abstract  symbols  can  be  made  arbitrarily  distinct”. 
Non-realistic  models  are  superior  to  symbols  for  “conveying 
category  (air  or  sea)  and  heading  information”.  [4] 

Another  advantage  of  symbols  is  aggregation  (discussed  in  more 
detail  later).  Aggregation  is  the  process  of  representing  a  set  of 
vehicles  as  a  single  entity.  A  simple  marking  on  the  top  of  the 
symbol  distinguishes  a  single  tank  from  a  platoon  of  tanks  or  even 
a  battalion. 

Billboards  do  have  some  drawbacks.  They  are  rotated  so  that 
they  always  face  the  point  of  view.  Consequently,  the  orientation 
of  the  vehicle  is  lost.  The  other  disadvantage  is  the  total  lack  of 
realism,  which  can  reduce  the  sense  of  immersion  and  hence 
enjoyment. 

So,  to  summarize,  a  realistic  view  should  be  used  when  vehicle 
details  are  1)  both  needed  and  possible  to  distinguish  given  the 
model  scale  and/or  2)  the  emotional  factors  of  a  realistic 
immersive  display  are  required.  A  non-realistic  model  provides 
quick  identification  of  force  affiliation  and  orientation  and  better 
performance,  but  not  identification  of  the  exact  vehicle  type. 
Symbolic  billboards  have  no  visual  or  emotional  appeal,  but  are 
very  descriptive  (except  for  orientation)  even  in  crowded  scenes, 
and  have  the  best  performance. 

3. 2. 2  Other  Alternative  Representations 
Terrain  can  be  represented  realistically  using  detailed  terrain 
database  models  and  satellite  imagery,  but  other  representations 
have  their  uses.  Two  techniques  emphasize  the  contours  of  the 
land:  shading  based  on  elevation  and  exaggerating  the  terrain. 
(See  the  color  plate  for  examples).  Elevation  shading  is  easier  to 
see  than  a  realistic  view.  Terrain  exaggeration  is  not  realistic,  but 
makes  it  easier  to  see  if  a  helicopter  is  higher  or  lower  than  a 
particular  mountain  peak  -  important  for  placing  communication 


equipment.  A  third  non-realistic  technique  is  draping  a  terrain 
model  with  political  and  road  maps.  This  helps  with  planning  and 
logistics  operations. 

Although  not  as  realistic  as  a  perspective  view,  using  an 
orthographic  view  to  determine  whether  something  is  left  or  right 
of  another  is  easier.  It  also  prevents  distant  objects  from 
“disappearing”  from  view. 

We  provide  an  “exaggerated  reality”  mode,  which  is  a  composite. 
The  terrain  and  non-realistic  vehicle  models  are  drawn  with 
perspective,  but  the  vehicles  are  automatically  scaled.  The  near 
and  far  vehicles  are  drawn  so  that  they  use  the  same  amount  of 
screen  real  estate.  This  provides  the  benefits  of  a  perspective 
display,  without  the  disadvantage  of  far  away  vehicles 
disappearing  from  awareness. 

There  are  several  techniques  for  drawing  shadows.  Gooch  et  al. 
discuss  three  modes:  “a  shadow  with  a  hard  umbra  and  hard 
penumbra,  a  single  hard  shadow,  and  a  soft  shadow”  [7].  In  non- 
realistic  mode,  we  exclusively  use  the  hard  shadow  since  it 
provides  clearer  identification  of  the  vehicle  shape,  and 
consequently  the  vehicle  type.  Unlike  Gooch  et  al.,  we  locate  the 
shadow  directly  “under”  the  vehicle,  independent  of  the  light 
source,  and  provide  an  altitude  line  from  the  center  point  of  the 
vehicle  model  to  the  center  point  of  the  shadow.  (Hence  the  term 
“drop- shadow”).  The  main  purpose  of  the  drop  shadow  is  to 
provide  information  about  the  XY  location  of  the  air  vehicle, 
which  may  not  be  obvious  in  a  perspective  view,  and  an 
additional  cue  as  to  the  vehicle  type.  Clearly  these  must  be  easy 
to  toggle  on  and  off,  as  they  have  the  potential  to  add  too  much 
clutter  to  the  screen,  the  same  reason  that  only  the  air-borne 
vehicles  are  given  drop  shadows. 

3.3  Data  Reduction 

Data  can  be  reduced  in  two  ways:  by  eliminating  it  completely, 
or  by  compressing  it  into  a  more  compact  format.  The  following 
sections  will  describe  one  instance  of  each. 

3. 3. 1  Data  Elimination 

Stone  et  al.  introduced  filters  that  can  be  used  to  reduce  some  data 
so  that  the  important  information  stands  out,  or  to  show  additional 
data  when  needed  [9].  For  example,  when  non-realistic  vehicle 
models  are  shown,  we  automatically  hide  vehicle  effects  such  as 
smoke,  fire,  dust  trails,  vapor  trails  etc.  This  allows  the  focus  to 
be  on  the  vehicle  itself.  Another  example,  which  we  have 
implemented  in  2D  but  not  yet  in  3D,  is  hiding  dead  enemy  units. 
This  emphasizes  the  enemy  units  that  are  still  a  threat.  An 
example  of  showing  additional  information  is  displaying  sensor 
volumes.  This  is  important  to  the  information  officer,  but  would 
just  add  unnecessary  clutter  to  the  display  of  the  logistics  officer. 

Our  filters  can  be  global  (apply  to  the  entire  screen),  or  local 
(apply  to  portion  of  the  screen,  and  change  between  these.  Filters 
can  be  easily  toggled  on  or  off.  Like  Stone  et  al.’s  work,  their 
effects  are  cumulative. 

Overlays  are  a  simplification  of  filters  for  a  particular  purpose. 
Tactical  graphics  are  symbols  which  users  place  on  a  battle  space 
map  to  assist  in  various  stages  of  a  military  operation.  These  are 
grouped  on  a  series  of  overlays.  Overlays  correspond  to 
traditional  acetate  sheets  overlaying  a  terrain  map,  upon  which 
one  can  draw.  Any  symbol  can  be  placed  on  an  overlay  layer. 
Through  the  overlay  manager,  overlays  can  be  dynamically 


created,  toggled  on  or  off,  or  even  reordered.  Unlike  acetate 
sheets,  overlays  can  be  shared  with  other  networked  colleagues. 

Other  techniques  to  reduce  the  data  visualized  are  user  controlled 
switches  to  turn  off  almost  anything  visualized,  dialogs  that  can 
be  closed  and  retrieved  at  will,  and  temporary  displays  of  data. 
For  example,  semi-transparent  information  dialogs  will  pop  up 
when  the  cursor  rolls  over  an  entity,  and  then  disappear  as  the 
cursor  moves  away.  This  provides  immediate  information, 
without  cluttering  the  display  with  dialogs  for  every  entity.  List 
boxes  provide  the  same  information,  but  organized  by  hierarchy, 
not  spatial  location.  In  addition,  multiple  entities  can  have  their 
associated  data  displayed  simultaneously. 

3.3.2  Data  Consolidation 

Another  important  difference  between  realistic  and  non-realistic 
views  is  the  level  of  detail.  Training  systems  are  not  just  first 
person  games,  with  individual  entities  such  as  tanks  represented 
as  realistically  as  possible.  Often,  due  to  the  size  and  level  of  the 
simulation,  vehicles  must  be  aggregated  into  larger  military 
groupings.  Military  commanders  are  often  interested  locally  in 
units  within  two  levels  of  their  own  level,  and  in  units  at  their 
level  in  adjoining  areas  of  interest  [10]. 

For  example,  a  brigade  commander  is  primarily  interested  in 
companies  and  above.  A  representative  battalion  might  have 
between  two  and  three  hundred  tanks  as  well  as  other  assorted 
ground  and  air  vehicles.  Due  to  the  scale  of  the  terrain  viewed,  it 
is  not  possible  to  depict  the  vehicles  to  scale.  If  they  were 
correctly  scaled,  they  would  not  be  visible.  If  scaled  larger  than  is 
realistic,  then  we  have  problems  with  vehicles  overlapping  when 
they  come  into  close  proximity  (for  example  a  battle  or  when 
traveling  as  a  convoy).  Consequently,  visually  representing  a 
group  of  vehicles  is  an  important  issue  for  training  systems. 

We  have  addressed  this  problem  through  two  techniques:  user 
controlled  dynamic  scaling  and  aggregation.  A  user  can  control 
the  scale  at  which  vehicles  are  drawn  though  a  simple  slider.  We 
examined  the  possibility  of  automatic  scaling  based  on  a 
combination  of  terrain  scale  and  entity  overlap,  but  found  that 
user  controlled  scaling  was  better  since  only  the  user  knew  what 
portion  of  the  screen  was  of  interest  at  a  given  moment.  There  are 
also  individual  differences  in  user  preference,  and  a  slider  to 
control  the  scale  is  also  better  in  this  regard. 

The  second  technique  to  deal  with  vehicle  scale  is  aggregation 
level.  The  aggregation  level  (level  of  detail)  applies 
independently  to  both  the  simulation  and  to  the  display.  Again, 
this  refers  back  to  the  emphasis  on  accurate  data  assessment  rather 
than  realism.  Vehicles  can  be  aggregated,  or  combined  into  a 
higher-level  representation.  For  example,  three  individual  tanks 
can  be  combined  into  a  squad;  three  squads  make  a  section;  three 
sections  a  platoon  etc.  Aggregation  levels  can  be  set  a  priori  or 
dynamically. 

Dynamic  aggregation  is  one  option  when  the  displayed  graphical 
models  overlap.  For  example,  if  the  models  are  scaled  larger  than 
real  life  (for  visibility  reasons),  then  when  a  subset  comes  into 
close  proximity,  they  overlap.  We  provide  three  options: 

•  Combine  only  those  vehicles  of  the  exact  same  type. 

•  Combine  overlapping  entities  with  identical 
Platform/Domain/Force  (e.g.  a  tank  and  truck  can  be 
combined  into  a  single,  more  general  'land'  visual) 


•  Combine  overlapping  entities  with  identical 
Platform/Force  (e.g.  a  tank  and  airplane  can  be 
combined  into  a  single,  more  general  'vehicles'  visual) 

3.4  Technique  Interaction 

Individual  3D  display  techniques  are  not  useful  to  an  application 
unless  they  are  easy  to  use.  They  must  work  both  with  other  3D 
display  techniques,  and  with  other  types  of  data  representations. 

3. 4. 1  Point  of  View 

It  is  important  that  the  users  can  easily  and  dynamically  choose 
the  view  that  is  most  informative.  The  user  can  easily  toggle 
between  realistic  and  non-realistic  modes;  and  between 
perspective  and  orthogonal  views.  The  standard  3D  controls  for 
setting  the  view  frustum  are  available.  These  settings  are  all 
independent,  so  each  combination  is  possible. 

We  allow  the  user  to  save  as  many  customized  views,  as  he/she 
wants.  These  views  consist  of  the  point  of  view,  and  the  terrain 
extents.  We  provide  the  two  most  common  views  in  the  saved- 
views  menu:  top-down  (2D)  view,  and  a  45 -degree  view.  Both  of 
these  views  are  initialized  based  on  the  extents  of  the  terrain. 

We  also  provide  cross  tool  control  mechanisms  to  allow  the  users 
to  control  the  3D  application’s  viewpoint  from  a  separate  2D 
application.  Our  2D  product  provides  additional  features  to  aid 
situational  awareness.  Many  of  these  features  would  not  be 
suitable  to  anything  except  a  top  down  2D  view.  Embedding 
them  in  the  3D  product  would  clutter  and  confuse  the  3D 
interface.  Likewise,  the  3D  views  provide  information  missing 
from  the  2D  views.  Sometimes  however,  only  one  view  is  needed 
for  particular  types  of  tasks.  Consequently,  these  remain  separate 
products.  However,  this  leads  to  the  requirement  for  cross-tool 
control. 

There  are  a  few  key  requirements  for  cross  tool  control.  First,  the 
2D  and  3D  products  must  work  together,  whether  they  are 
running  on  the  same  machine  or  not.  Not  just  technically,  but  with 
an  easy  to  use  and  understand  interface.  Second,  a  single  2D 
application  must  be  able  to  control  multiple  3D  applications 
views. 

On  the  2D  display,  a  pair  of  binoculars  icon  represents  each 
controlled  3D  view.  The  3D  view  can  be  attached  to  individual 
vehicles  or  sets  of  them.  This  means  the  view  moves  as  the 
relevant  vehicle(s)  do.  A  line  between  the  binocular  icon  and  the 
vehicle  icon  represents  the  relative  offset  and  eye  point  (which 
need  not  be  looking  directly  at  the  associated  vehicles).  There  is 
substantial  flexibility  in  manipulating  the  3D  view,  but  we  will 
not  go  into  the  details  here. 

The  3D  controller  icons  appear  in  the  hierarchical  list  view  of 
entities.  This  allows  the  user  to  quickly  find  the  controllers,  even 
when  the  main  2D  view  is  focused  on  a  subset  of  the  terrain 
which  does  not  contain  the  controllers. 

3. 4. 2  Other  Data  Representations 

Sometimes  a  spatial  view  is  not  the  best  information  format, 
whether  it  is  realistic  or  non-realistic.  For  some  types  of  data, 
both  the  spatial  and  non-spatial  data  representation  offer  things 
the  other  cannot,  and  processes  for  using  both  concurrently  must 
be  developed. 

Vehicle  representation  is  one  such  type  of  data.  Clearly  the  best 
depiction  of  a  vehicle’s  position  relative  to  other  vehicles  and  the 


terrain  features  is  spatial.  But  to  represent  the  vehicles 
organizational  structure,  a  hierarchical  list-view  is  best.  The  user 
needs  to  be  able  to  locate  a  particular  vehicle  in  one 
representation  from  the  other  etc. 

Another  example  of  this  type  of  data  is  the  sensors  that  can  be 
attached  to  vehicles.  Placing  them  in  the  organizational  hierarchy 
as  subcomponents  of  their  vehicle  allow  the  user  to  “attach”  to 
them,  and  monitor  their  details  through  information  dialogs. 


Figure  3  An  F16  (having  ID  1:303:1)  is  emitting  a  sensor.  Its 
sensor  has  one  beam  whose  azimuth  sweep  extends  3  degrees 
and  whose  elevation  sweep  extends  20  degrees.  The  sensor’s 
attributes  are  described  both  visually  (through  the  pink 
volume  emanating  from  the  nose  of  the  jet)  and  textually 
(through  the  information  dialog  to  the  right). 

3. 4. 3  Reducing  Confusion 

There  are  many  things  that  are  best  represented  as  lines.  But  too 
many  lines  create  confusion,  not  understanding.  For  example, 
history  trails,  fire  and  detonate  lines,  communication  links,  lines 
of  sight,  roads,  routes,  tactical  graphics  etc.  Clearly  the  ability  to 
hide  lines  by  category  is  required.  One  can  also  use  different 
types  of  lines:  different  colors,  dashed/solid,  fine  line  /  ribbon  / 
tubes,  arrows  on  the  ends  etc.  Since  different  users  are  interested 
in  different  subsets  of  data,  we  allow  users  to  easily  change  the 
default  configuration  for  each  data  category. 

4.  RELATED  PRODUCTS 

When  we  look  at  related  work,  we  are  primarily  interested  in 
other  situational  awareness  (SA)  products  that  are  of  commercial 
quality  and  available  as  retailed  products,  i.e.  commercial  off-the- 
shelf  products  or  COTS  products.  Our  3D  visualization  product  is 

called  the  MAK  StealthXRtm. 

Analytical  Graphics  makes  Satellite  Toolkit  (STK)[1 1].  There  are 
three  major  differences  between  STK  and  the  StealthXR: 
underlying  data  transport  mechanism,  scope,  and  cost.  STK  does 
not  support  HLA  or  DIS  (the  network  protocols  used  for  many 
military  applications).  STK  focuses  on  an  earth  view,  showing 
cones  of  coverage  as  opposed  to  closer  coverage  suited  to 
battlefield  vision  at  the  level  of  planes  and  tanks.  It  is  also  seven 
times  as  expensive  as  the  StealthXR 


eNGENUlTY  makes  VAPS[12].  These  animated  instrumentation 
components  deal  with  instrumentation,  not  visualization.  For 
example,  data  input  from  outside  stimulus  is  used  to  control  the 
needle  display  on  a  graphical  cockpit-dial.  These  instruments  are 
primarily  oriented  at  helicopter  and  aircraft  applications.  It  is 
primarily  a  code  generator  rather  than  extensible  toolkit  or  part  of 
a  tactical  training  system. 


DiSTI  (distributed  simulation  technology,  inc.)  makes  GL  Studio 
[13].  GL  Studio  synthesizes  photographs,  3D  models  and 
x  behavior  logic  to  create  photo-realistic  3D  interactive  real-time 
Reusable  Simulation  Objects.  Like  VAPS,  these  are  primarily 
animated  instrumentation. 


5.  RESULTS  AND  FUTURE  WORK 


MAK’s  StealthXR  visualization  product  was  demonstrated  and 
has  received  positive  feedback  from  many  tradeshows  and 
military  research  conferences,  including:  Simulation 
Interoperability  Workshop  (SIW)  2004,  International  Training 
Education  Conference  (ITEC)  2004,  C4I  Summit  2004  and 
Transformation  and  Operations  (TOPS)  in  Cyberspace  2004 


The  Air  Force  contract  under  which  this  work  has  been  done  is 
still  not  finished.  Despite  this,  the  StealthXR™  toolkit  (which 
encompasses  the  work  described  in  this  paper)  has  two  customers. 
It  has  been  successfully  used  as  a  base  toolkit  by  OpNet  to  create 
a  communications  visualization  tool  called  3DNV  ™.  OpNet  in 
turn  has  generated  sales  from  their  new  product.  Our  second 
customer,  ITT,  has  used  the  toolkit  to  visualize  hazardous  clouds 
which  are  driven  by  data  they  created  using  sophisticated  particle 
based  mathematical  models. 

This  paper  describes  a  commercial  product,  whose  on-going 
development  and  maintenance  will  continue  for  many  years. 
Many  of  our  plans  are  based  on  preliminary  customer  feedback 
and  trade  show  demonstrations,  but  will  change  with  additional 
feedback.  The  future  work  plans  are  too  lengthy  to  list  here,  so  I’ 11 
just  mention  a  few  of  the  major  categories. 


User  studies.  Many  of  the  implemented  features  were  created 
with  feedback  from  knowledgeable  about  military  needs. 
However,  objective  productivity  measures,  obtained  through 
subject  experiments  would  provide  invaluable  feedback. 


Performance.  For  example,  convert  the  symbolic  billboard 
models  to  fonts  blitted  onto  a  billboard.  We  do  this  in  our  2D 
product,  and  previous  performance  testing  showed  that  true  type 
fonts  were  up  to  an  order  of  magnitude  faster  than  comparable 
bitmaps.  Also,  improved  outline  techniques  for  the  non-realistic 
vehicles  models.  Currently  this  is  based  on  work  by  Gooch  et  al. 
[8],  Lake  et  al.  [14]  and  Buchanan  and  Sousa  [15],  but  our 
implementation  could  be  more  general. 


Better  data  filtering.  For  example,  only  enemy  units  within 
striking  distance,  show  only  munitions  (missiles,  torpedoes, 
bombs),  only  vehicles  within  a  certain  distance  of  a  point,  or 
between  particular  latitude/longitude  lines. 


Automated  data  displays.  Spawning  sub- windows  based  on 
activity  (pop  up  a  window  focused  on  an  area  under  attack) 


Knowledge  voids.  One  of  the  most  extreme  examples  of 
"inadequate  information"  is  no  information  at  all,  or  a  knowledge 
void.  By  simply  and  easily  identifying  regions  of  the  battle  space 
where  we  do  not  have  any  data,  commanders  can  make  more 


informed  decisions  about  where  to  place  sensors  or  send 
information  gathering  sorties. 
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Color  Plate  1:  Raster  map  draped  over  terrain,  non- 
realistic  models. 
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Color  Plate  2:  Terrain  shading,  drop  lines,  red  (OPFOR) 
information  dialog,  and  sensor  volume. 


Color  Plate  3:  Correctly  scaled  terrain. 


Color  Plate  4:  Same  terrain  --  exaggerated  scaling. 


Color  Plate  6:  Visualization  of  communication  lines. 


Color  Plate  5:  Visualization  of  non-visual  data  like 
Nuclear,  Biological,  or  Chemical  clouds 


